<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.12.0"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>NeuZephyr: nz::opt Namespace Reference</title>
<link rel="icon" href="NZ_logo2.png" type="image/x-icon" />
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectlogo"><img alt="Logo" src="NZ_logo2.png"/></td>
  <td id="projectalign">
   <div id="projectname">NeuZephyr
   </div>
   <div id="projectbrief">Simple DL Framework</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.12.0 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() { codefold.init(0); });
/* @license-end */
</script>
  <div id="navrow1" class="tabs">
    <ul class="tablist">
      <li><a href="index.html"><span>Main&#160;Page</span></a></li>
      <li><a href="pages.html"><span>Related&#160;Pages</span></a></li>
      <li class="current"><a href="namespaces.html"><span>Namespaces</span></a></li>
      <li><a href="annotated.html"><span>Classes</span></a></li>
      <li><a href="files.html"><span>Files</span></a></li>
    </ul>
  </div>
  <div id="navrow2" class="tabs2">
    <ul class="tablist">
      <li><a href="namespaces.html"><span>Namespace&#160;List</span></a></li>
      <li><a href="namespacemembers.html"><span>Namespace&#160;Members</span></a></li>
    </ul>
  </div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function(){ initResizable(false); });
/* @license-end */
</script>
<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><b>nz</b></li><li class="navelem"><a class="el" href="namespacenz_1_1opt.html">opt</a></li>  </ul>
</div>
</div><!-- top -->
<div id="doc-content">
<div class="header">
  <div class="summary">
<a href="#nested-classes">Classes</a>  </div>
  <div class="headertitle"><div class="title">nz::opt Namespace Reference</div></div>
</div><!--header-->
<div class="contents">

<p>Contains optimization algorithms for training deep learning models.  
<a href="#details">More...</a></p>
<table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="nested-classes" name="nested-classes"></a>
Classes</h2></td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_ada_delta.html">AdaDelta</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="classnz_1_1opt_1_1_ada_delta.html" title="AdaDelta optimizer for deep learning models.">AdaDelta</a> optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_ada_delta.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_ada_grad.html">AdaGrad</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="classnz_1_1opt_1_1_ada_grad.html" title="AdaGrad optimizer for deep learning models.">AdaGrad</a> optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_ada_grad.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_adam.html">Adam</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="classnz_1_1opt_1_1_adam.html" title="Adam optimizer for deep learning models.">Adam</a> optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_adam.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_momentum.html">Momentum</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="classnz_1_1opt_1_1_momentum.html" title="Momentum optimizer for deep learning models.">Momentum</a> optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_momentum.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_n_adam.html">NAdam</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="classnz_1_1opt_1_1_n_adam.html" title="NAdam optimizer for deep learning models.">NAdam</a> optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_n_adam.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_optimizer.html">Optimizer</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Base class for optimization algorithms in deep learning.  <a href="classnz_1_1opt_1_1_optimizer.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_r_m_sprop.html">RMSprop</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="classnz_1_1opt_1_1_r_m_sprop.html" title="RMSprop optimizer for deep learning models.">RMSprop</a> optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_r_m_sprop.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">class &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="classnz_1_1opt_1_1_s_g_d.html">SGD</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Stochastic Gradient Descent (<a class="el" href="classnz_1_1opt_1_1_s_g_d.html" title="Stochastic Gradient Descent (SGD) optimizer for deep learning models.">SGD</a>) optimizer for deep learning models.  <a href="classnz_1_1opt_1_1_s_g_d.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table>
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
<div class="textblock"><p>Contains optimization algorithms for training deep learning models. </p>
<p>The <code><a class="el" href="namespacenz_1_1opt.html" title="Contains optimization algorithms for training deep learning models.">nz::opt</a></code> namespace includes a collection of optimization algorithms designed to update model parameters during the training of deep learning models. These optimizers aim to minimize the loss function by adjusting the learning rate dynamically or incorporating momentum terms to improve convergence.</p>
<p>Key components in this namespace:</p><ul>
<li><b><a class="el" href="classnz_1_1opt_1_1_s_g_d.html" title="Stochastic Gradient Descent (SGD) optimizer for deep learning models.">SGD</a> (Stochastic Gradient Descent)</b>: A basic optimization method that updates model parameters in the direction of the negative gradient, with a fixed learning rate.</li>
<li><b><a class="el" href="classnz_1_1opt_1_1_momentum.html" title="Momentum optimizer for deep learning models.">Momentum</a></b>: Enhances <a class="el" href="classnz_1_1opt_1_1_s_g_d.html" title="Stochastic Gradient Descent (SGD) optimizer for deep learning models.">SGD</a> by introducing a momentum term that helps accelerate convergence and reduces oscillations.</li>
<li><b><a class="el" href="classnz_1_1opt_1_1_ada_grad.html" title="AdaGrad optimizer for deep learning models.">AdaGrad</a></b>: An optimizer that adjusts learning rates based on the historical gradients, allowing it to handle sparse data more effectively.</li>
<li><b><a class="el" href="classnz_1_1opt_1_1_r_m_sprop.html" title="RMSprop optimizer for deep learning models.">RMSprop</a></b>: A modification of <a class="el" href="classnz_1_1opt_1_1_ada_grad.html" title="AdaGrad optimizer for deep learning models.">AdaGrad</a> that uses a moving average of squared gradients to stabilize the learning rate, leading to more consistent updates.</li>
<li><b><a class="el" href="classnz_1_1opt_1_1_adam.html" title="Adam optimizer for deep learning models.">Adam</a> (Adaptive Moment Estimation)</b>: Combines the benefits of momentum and <a class="el" href="classnz_1_1opt_1_1_r_m_sprop.html" title="RMSprop optimizer for deep learning models.">RMSprop</a>, providing adaptive learning rates for each parameter using first and second moment estimates.</li>
<li><b><a class="el" href="classnz_1_1opt_1_1_n_adam.html" title="NAdam optimizer for deep learning models.">NAdam</a> (Nesterov-accelerated <a class="el" href="classnz_1_1opt_1_1_adam.html" title="Adam optimizer for deep learning models.">Adam</a>)</b>: An improvement over <a class="el" href="classnz_1_1opt_1_1_adam.html" title="Adam optimizer for deep learning models.">Adam</a> by incorporating Nesterov momentum, which helps achieve faster convergence.</li>
<li><b><a class="el" href="classnz_1_1opt_1_1_ada_delta.html" title="AdaDelta optimizer for deep learning models.">AdaDelta</a></b>: A variant of <a class="el" href="classnz_1_1opt_1_1_ada_grad.html" title="AdaGrad optimizer for deep learning models.">AdaGrad</a> that maintains a constant learning rate by using a running average of squared updates, avoiding the diminishing learning rate problem.</li>
</ul>
<p>These optimizers are designed to work efficiently in high-performance computing environments, utilizing GPU-based tensor operations to accelerate training. The algorithms in this namespace can be easily extended to support additional optimization strategies in the future.</p>
<p>This namespace plays a critical role in the optimization of deep learning models by providing a set of tools to adaptively adjust model parameters during training, improving the overall performance and stability of the training process.</p>
<dl class="section note"><dt>Note</dt><dd>The optimizers in this namespace rely on tensor-based operations for efficient computation. Ensure that proper memory management and error handling are applied when using these algorithms.</dd></dl>
<dl class="section author"><dt>Author</dt><dd>Mgepahmge(<a href="https://github.com/Mgepahmge">https://github.com/Mgepahmge</a>)</dd></dl>
<dl class="section date"><dt>Date</dt><dd>2024/12/07 </dd></dl>
</div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.12.0
</small></address>
</div><!-- doc-content -->
</body>
</html>
